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ABSTRACT 

Assessing learners’ performance is crucial as it informs about learners’ achievement levels in 
addition to their strengths and weaknesses, not only in face-to-face but also in online teaching. 
The study aimed at investigating the challenges faced by language teachers in assessing learners 
during online teaching and to offer some suggestions regarding language assessment courses in 
teacher training programs. The study adopted a qualitative approach to data collection and analy- 
sis. The participants included 22 language teachers aged between 24 to 48. Of these teachers, 
8 were male, while the rest were female. Their teaching experience varied from 2 to 25 years, 
with an average of 11.8 years. The data was based on the responses of the participants during 
semi-structured interviews. The participants were asked to attend ZOOM meetings to be held, 
considering their availability. The interviews lasted 10 to 25 minutes. The results of the study in- 
dicated that the challenges faced by the participants included five major themes: Assessment type, 
assessment item formats, support, previous training (assessment literacy), and academic integrity. 
Several suggestions were offered considering online testing and assessment literacy and language 
assessment courses in teacher training programs. 
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1. Introduction 

Assessment practices in learning and teaching contexts are an indispensable 
part of any teaching and learning context, and language teaching is no exception 
since it includes gathering information on what students or learners know and 
how they perform based on their educational experience. Case and Obenchain 
(2010) stress the importance of observing and assessing students throughout the 
semester and informing students of their performance to focus on the weaknesses 
and strengths, which is also known as formative assessment. Formative assess- 
ments are planned within a face-to-face or online course or lesson and aimed 
at determining learner weaknesses and strengths, leading to acting accordingly 
through some remedial practice or tutorial. The main function of this type of as- 
sessment is that it is ongoing, consistent, and provides critical feedback to learn- 
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ers, while summative assessments are conducted at the end of a semester or after 
several weeks to measure what learners have learned (Absolum, Flockton, Hat- 
tie, Hipkins, & Reid, 2009). This type of assessment is mainly used to indicate 
whether or not overall learning goals have been achieved. Formative and spe- 
cifically summative assessments are mainly conducted through paper-and-pencil 
tests through which learners’ grammar and vocabulary knowledge are assessed. 
Traditional assessment instruments benefiting from multiple-choice, and fill-in- 
the-blank are the main assessment items that focus on discrete-point assessments 
(Brown & Abeywickrama, 2010; Hughes, 2003). In addition to traditional ways 
of assessment, teachers might also consider alternative assessment methods such 
as projects, self and peer assessment along with the traditional written assessment 
(Brumen, Cagran, & Nixon, 2009). As Falsgraf (2009, pp. 495-496) indicates, 
there are several obstacles to appropriate and healthy assessment. 

Most language teachers are enrolled at their department during their educa- 
tion on at least one course on language assessment and evaluation. One obstacle 
is that teachers generally may not receive the necessary training in assessment 
practices, leading to a lack of knowledge and practice in choosing the most ap- 
propriate instruments. Buytikkarci (2014), in his investigation of language teach- 
ers’ assessment beliefs and their practices working at primary schools in Turkey, 
found that although teachers have positive attitudes and beliefs about formative 
assessment of students’ performances, they do not apply formative assessment 
practices. The main obstacles found to account for this are crowded/overcrowded 
classrooms and teachers’ workload. Another similar study conducted by Han and 
Kaya (2014) has investigated Turkish EFL teachers’ assessment preferences and 
practices at primary and secondary educational levels. The results show that the 
teachers focus more on reading and speaking skills than on listening and writing 
skills. Moreover, it was found that half of the teachers consider the integration of 
the language skills important in teaching contexts. Gender and previous training 
on assessment are not found to be variables that affect assessment preferences. 
Glover (2014) analysed the effects of examinations on teacher talk through dis- 
course analysis. Findings suggest that teacher talk may be affected by examina- 
tions as the teachers in the study tend to move away from the role of guiding to 
telling in their talks. The findings also indicated that teachers feel the force to 
match their regular teaching with the format of the examination to better prepare 
students to score high on examinations. They also reported negative effects on 
learner motivation due to the burden and stress on the learners. Compared to the 
core knowledge about language testing such as types of testing, reliability, and 
validity, the pre-service language teachers who were the participants of the study 
conducted by Hatipoglu (2010) expressed the need for more time for practising 
writing items for the language skills and evaluating a variety of exams conducted 
in various situations. 
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Considering the findings of the above studies, it might be stated that teacher 
training programs prepare teachers, more or less, for assessments in the face-to- 
face contexts, with no or little specific attention to online assessment practices 
although there might be several personal attempts to integrate technology-based 
assessments into the curriculum or courses. The major challenge during the pan- 
demic was that language teachers did not have the chance to meet their students 
face to face in the physical classroom. Therefore, they could not move the face-to- 
face assessment practices into online teaching environments and tried to manage 
by themselves, especially at the very beginning of the pandemic. This study, there- 
fore, aimed at determining the challenges faced by language teachers regarding 
assessment during the pandemic and to offer some suggestions as to the content of 
the language assessment courses in teacher training during and after the pandemic. 


2. Material and Methods 


2.1. Research context and participants 

The study included public high schools as the research context in Turkey. The 
teachers working at high schools were contacted on a Facebook group. This in- 
cluded around fifty thousand language teachers who were working at various 
schools from primary to university contexts. 22 teachers responded to the message 
shared on the group explaining the purpose of the study and agreed to participate 
in the study through semi-structured interviews. The teachers were aged between 
24 to 48. Of these teachers, 8 were male, while the rest were female. Their teach- 
ing experience varied from 2 to 25 years, with an average of 11.8 years. 


2.2. Data collection and procedure 

The study adopted a qualitative approach to the data collection process to inves- 
tigate the possible answers to the research questions of the study. The data was 
based on the responses of the participants during the semi-structured interviews. 
The participants were asked to attend ZOOM meetings to be held, considering 
their availability. The interviews lasted 10 to 25 minutes and were not recorded 
as the participants did not agree. As the recording was not possible, the researcher 
tried to focus on the responses by taking notes. All the interviews were conduct- 
ed in Turkish, and the following questions were directed to the participants: 1) 
How have you assessed your students during your online teaching? 2) Have you 
used any software or website to assess your students? 3) What has been the main 
purpose for assessing your students? 4) Has your school provided you with any 
technical or training support? In what ways? 5) Do you think that your university 
education has prepared you for assessing students during online teaching? In what 
ways? 6) What are the issues and opportunities that you have faced during your 
online teaching? 7) Do you have any other comments/suggestions? 


48 Ferit Kiligkaya 


2.3. Data analysis 

The data collected through the semi-structured interviews were subject to induc- 
tive content analysis. The notes taken from the interviews were re-read repeatedly 
by the researcher to determine the common themes and codes. Codes can be de- 
fined as the labels to determine recurring topics, issues, or themes in the qualita- 
tive data and to characterize particular perceptions. These themes and codes were 
hierarchically ranked. The emerging themes and sub-categories were checked 
against consistency by another expert in the field, with ample knowledge of induc- 
tive content analysis. Every effort has been made to provide compelling examples 
to demonstrate the themes and codes and to maintain a rich overall description; 
however, it can be considered quite natural to lose some depth and complexity 
when the analysis is to be provided in an article with strict word limitations. This 
is especially valid since it is difficult to provide a convincing analysis by just pro- 
viding adequate examples as one or two extracts for a theme or a code. 


3. Results 

The themes and the codes, together with the exemplar responses, have been pro- 
vided in Table 1. The table also includes selected quotations as a representative of 
the responses to the interview questions. 


Table 1. Themes and codes obtained from the responses 


Themes Codes Examples 


“T have tried to assess my learners every two weeks or 
so to determine the problem areas so that I can focus 
on those issues during my online teaching in the next 

classes”. 


Formative 


Assessment type 


“Tt was almost impossible to assess my students’ 


Summative : ; : : 
academic achievement as the regulations did not allow 
that”. 
Traditional “My assessment was asynchronous. I mean, I just 
(selected assigned assignments on EBA. My students answered 
response) multiple-choice questions and matching activities to 
: review the course content”. 
Assessment item 
format : : 
. : Alternative “T hardly used alternative assessments as my students 


assessment did not have the necessary technological devices. Some 
(constructed |used only mobile phones, while some others had laptops 
response) and tablets”. 
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Themes Codes Examples 


“My school did not provide any technical support 
Technical when we had problems connecting to EBA to have 
synchronous classes. I asked my friends or other 
Support teachers to help me”. 

No training has been provided. I just watched the 
Training videos shared on the school website or any other videos 
suggested by my friends and colleagues”. 


“T know how to assess the basic language skills and 
language components though I need more practice in 
creating and choosing appropriate assessment items”. 


Assessing skills 


Previous training 


assessment literac : : ‘ : = 
( y) “T think we were not provided with detailed training and 


Online sa ; : : : 

aseaucrnBie practice in conducting online assessments, especially in 
alternative assessments”. 

In-secure “Summative assessment or high-stakes exams were not 

assessment possible as cheating or getting others’ help is easy”. 

Academic integrity 
“Sometimes I cannot be sure whether the work is my 
Authorship student’s work”. 


Assessment type 
Regarding assessment types, although the participants did not state “formative” 
and “summative”, the responses (n= 18) indicated that the participants benefited 
from formative assessment through multiple-choice and matching items. They 
benefited from several tools and websites such as Quizziz, Worldwall, and Live- 
worksheets both to create and to use ready-to-use materials which are created by 
other teachers and users. Several less frequently used assessment tools were also 
named, such as Kahoot and Mentimeter. However, summative assessment seemed 
to be a genuine concern as scoring and grading practices were determined by the 
Ministry of Education, and face-to-face exams were required, although it was 
not possible to have these exams. One of the participants expressed this in this 
way: “I know very well that my students also need some communicative ability in 
English. Although I try to integrate some listening and speaking activities in my 
assessment, it was not possible as attending online classes was not possible due 
to two main reasons. The first one was that attendance was not mandatory. Some 
students simply did not have the necessary equipment and access to the Internet. 
The assessment was often not possible, as a result” (Interviewee 12, Female). 
Another participant underscored the importance of frequent testing saying that: 
“T believe that frequent testing and providing feedback help my students notice 
their weaknesses. As a result, every two weeks | give my students mini online 
exams including multiple-choice questions because this is the format of the test. 
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But for the official examinations, I also use gap-filling exercises, especially for 
grammar” (Interviewee 29, Male). 


Assessment item formats 

Based on the responses, it was determined that the participants mainly created and 
used traditional (selected-response) assessment item formats such as matching, 
multiple-choice, and fill-in-the-blank activities. These activities were mainly as- 
signed and conducted asynchronously. In other words, live feedback was not pos- 
sible. Some (n=14) created activities based on gamification using websites such as 
Wordwall. However, the number of these activities, according to the participants, 
was few. Almost all the participants (n=20) raised serious concerns regarding al- 
ternative assessment. Considering the responses, while it is possible to say that 
most of the participants excelled at supplementing the lessons with authoring or 
using digital assessment tools, conducting alternative assessments online seemed 
to be impossible for the participants. Related to this, two of the participants stated 
that: “Each week I assign some online worksheets using the website Livework- 
sheets to my students. These worksheets include dialogues together with some vo- 
cabulary activities based on these dialogues. Sometimes they do their homework 
at home, and sometimes we check the answers during online teaching. But I could 
not use or, honestly speaking, do not know how to conduct an online alterna- 
tive assessment” (Interviewee 8, Female). “I very well know that multiple-choice 
questions are just based on recognition and do not test students’ productive skills. 
However, the tools and websites available do not allow me to create exercises 
requiring them to produce” (Interviewee 13, Female). 


Support 

A great majority of the participants (n=17) indicated that the technical and training 
support was not sufficient, especially at the very beginning of the pandemic. They 
also added that they were not provided with any training regarding teaching and 
learning online, including assessment, except several videos that either they found 
themselves or that were provided to them. One participant indicated this lack of 
support saying that: “I felt alone when I had connection problems or needed sup- 
port on how to use specific software or hardware. For example, I tried to solve 
my connection problems to EBA [an online platform for all teachers and students 
which includes online activities, exams, and tests]. We did not have any specific 
training on teaching or assessing skills online but attended several seminars on 
Facebook and YouTube” (Interviewee 7, Female). 
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Previous training (assessment literacy) 

Regarding training on assessment literacy during teacher training programs, most 
of the participants (n=15) stated that they had several courses during their uni- 
versity education in teacher training programs on language assessment principles 
such as reliability and validity, writing items, and assessing skills. However, it 
was indicated that despite their previous training, they still felt the need to practice 
creating and/or choosing appropriate assessment items and scoring alternative as- 
sessment instruments. Almost all the participants (n=21) pointed to the fact that 
they did not receive any comprehensive training on how to assess language skills 
and components in online or distance education environments. One participant 
explained this as follows: “In my opinion, I know more or less how to assess the 
basic language skills and language components though I need more practice in 
creating and choosing appropriate assessment items. I believe that we were not 
provided with detailed training and practice in conducting an online assessment, 
especially in alternative assessment” (Interviewee 5, Female). 


Academic integrity 

As for summative assessment and high-stakes exams through which grades are 
finalized, the majority of the participants raised concerns about the security of 
assessment and authorship. In addition to technical constraints and limitations, 
the participants indicated that the assessment policy required by the Ministry of 
National Education also kept them from utilising alternative assessments. They 
indicated that they had the knowledge and practice to use various alternative as- 
sessments such as assignments, projects, portfolios, and discussions. However, 
several concerns were voiced, such as the security of assessment, as it would not 
be possible to ensure that the students created the work on their own or without 
any unethical help or support from others. One participant indicated that: “Sum- 
mative assessment or high-stakes exams were not possible as cheating or getting 
others’ help is easy. Online quizzes cannot also be utilized as again technically it 
would not be possible to secure the exams. Sometimes I cannot be sure whether 
the work is my student’s work” (Interviewee 3, Male). 


4. Discussion and conclusion 

Based on the responses, it can be stated that during the pandemic, formative assess- 
ment received more attention from the participants due to several reasons. One is 
that the participants were well aware that during online teaching and the pandemic, 
it was more crucial than ever to determine learners’ strengths and weaknesses. In 
line with what is stressed by Folse, Hubley, and Coombe (2007), Green (2021), and 
Uzun and Ertok (2020), the participants highly valued formative assessment during 
online teaching. However, summative assessment seems to have failed during on- 
line teaching especially when it serves in the form of high-stakes exams. Therefore, 
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it can be put forward that although summative assessment plays a significant role, 
online teaching practices lack the essential benchmark to check learners’ progress, 
schools, and educational programmes. This might also mean that online teaching 
and assessment did not benefit from beneficial washback on learners (Cakir, 2017; 
Gastaldi & Grimaldi, 2021; Kilickaya, 2016; Oztiirk-Karatas & Okan, 2019, 2021). 

Regarding assessment items, it was also seen that assessing learners were lim- 
ited to grammatical forms, vocabulary, and to some extent reading comprehension 
through selected-response items such as multiple-choice questions and matching, 
which is in line with findings of several other studies (Bttyikkarci, 2014; Golver, 
2014; Han & Kaya, 2014). This seems to be mainly due to the technical resources 
available and the features of these resources and websites such as providing im- 
mediate feedback regarding correct and incorrect answers (Ferdig, Baumgartner, 
Hartshorne, Kaplan-Rakowski, & Mouza, 2020; Fitriyah & Jannah, 2021; Paudel, 
2021). The participants mainly opted for tools and websites which provide easy and 
free access to online assessment tools such as Quizlet and Wordwall, and Edpuzzle. 
This assessment practice seems to have resulted in neglecting assessment of com- 
munication and the ability to comprehend authentic materials. In addition to this, 
engaging in person-to-person communication in Zoom meetings and presenting 
topics or ideas were seldom practised by the learners during these online teaching 
practices. 

The responses indicated that the participants received little, if not any, techni- 
cal support and training when they had problems using the platform required by 
the Ministry of Education. The participants sought help and solutions from You- 
Tube videos including some sort of training regarding the challenges they faced 
such as troubleshooting on technological devices and Internet connections, which 
appeared as the common challenges faced in online teaching as displayed by Can- 
polat and Yildirim (2021), and Fitriyah and Jannah (2021). This indicates that the 
participants lacked official support from the schools, although there were some 
seminars and workshops directed towards the general audience. 

The participants’ previous training in language assessment and evaluations 
prove that the participants had some experience in assessing language skills and 
components, such as grammar and vocabulary. However, the real challenge ap- 
peared to be related to assessing these skills online using the tools available. Most 
believed that they needed more practice in creating and choosing appropriate as- 
sessment items, which is consistent with findings of several other studies (Bru- 
men et al., 2009; Hatipoglu, 2010; Olmezer-Oztiirk, Oztiirk, & Aydin, 2021). The 
responses of the participants also indicated the need for training in online assess- 
ment and several websites and software to utilise online formative and summative 
assessment. It was made clear by the participants that individual efforts of the staff 
at the teacher training programmes, and the number of the courses could not be 
sufficient and efficient. Therefore, a carefully designed program including more 
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practice and applications in assessment in both in teaching face to face and online 
classrooms is recommended. This could also be achieved by integrating assess- 
ment into school experience and practice teaching courses as part of the training 
program (Hidri, 2021). 

Academic integrity was another theme that emerged during the analysis of the 
responses leading to two codes: insecure assessment and authorship. Consider- 
ing the realities of online assessment, it might be quite natural that maintaining 
academic integrity would appear challenging and maybe even impossible. Imple- 
menting online exams with selected-response items is often more prone to cheat- 
ing than written or spoken tasks (Harper, Bretag, & Rundle, 2020). Most teachers, 
like the ones in the current study, are willing to use alternative assignments, such 
as reflection tasks. Several participants showed interest in take-home exams not 
only to benefit from the advantages of summative assessment but also to reduce 
students’ anxiety (Bengtsson, 2019); however, they added that since they were 
wotried about the authorship and some other technical issues; they decided not to. 

In conclusion, the study tried to investigate language teachers’ experience with 
online language assessment practices and their views regarding the challenges 
faced. Their responses indicated that they needed more training, practice, and sup- 
port on utilizing online assessment tools, with specific needs on alternative as- 
sessment as, to some extent, they excelled at using traditional assessment items 
via several websites and tools available. Therefore, the results also indicated the 
need for reconsidering language assessment courses in teacher training programs 
during and after the pandemic as even when the pandemic is over and its effects 
are lessened, online assessment practices would be an integral part of our teaching 
and learning practices (Russell & Murphy, 2021). 

As with any practical research, it is worth noting that there are several limi- 
tations of the study in various aspects. It must be acknowledged that the study 
utilized the participants’ responses in data analysis and their reported experiences 
were considered in data analysis, presentation of findings, and the discussion of 
these findings. Therefore, readers might like to take into account that what was 
reported by the participants might well be different from what they do or did in 
their teaching contexts. Therefore, further research focus on observations of on- 
line assessment practices through the example instruments as used by the partici- 
pants. As a form of data triangulation, the participants could also be asked to keep 
a journal in which they write down their feelings and experiences on a daily basis. 
Furthermore, in addition to teachers’ perspectives, learners’ experience with on- 
line assessment and their views on academic integrity should also be investigated. 
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